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. (S/NF/SS-2) EVALUATION PROCEDURES: 
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1. (S/NF/SS-2) Operational Phase: 


(S/NF/SS-2) The Project Manager will assess the value of the operational 
phase of the SUN STREAK program by requiring all data consumers (i.e., 
operational task originators) to furnish him with data accuracy and data 
utility assessments. Table 1 illustrates the general format and 
assessment scale (0-3) for this evaluation. Specific data categories may 
vary depending on the nature of the task or on specific objectives; 
however, the overall approach will remain the same. 


(S/NF/SS-2) The Project Manager will enter results of the accuracy and 
utility assessment into the project data base. He will make periodic 
reviews (quarterly) of this data to assess overall effectiveness. 


(S/NF/SS-2) The Project Manager, in coordination with the task 
originators and the Intelligence Community Task Coordinating Group, wil} 
develop an additional measure of program value. This measurement 
(overall program value) is designed to measure the net worth or total 
contribution of the task in relation to other tasks and to the overall 
intelligence mission. Data accuracy and utility may be high for a 
particular consumer, but the results may not have a significant overall 
impact. This program value scale is similar to the utility scale; 
however it is based on specific measures of over-all benefit. This value 
scale is as follows: is 


Program Value Scale 


0 - Little or no over-all program value, 
1 - Some program value (e.g., helped refine estimates), 
2 - Moderate value (e.g., identified new data, narrowed down 


possibilities), 


3 - High value (e.g., led to significant cost savings, 
identified critical high priority S&T or operational activities), 


4 - Exceptional value (e.g., predicted major 
events/activities, located lost/missing resources or personnel). 


The value evaluation will be used in conjunction with other program 


evaluation parameters (accuracy, utility) for determining program accom- 
plishments and for defining future program directions. 
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2. (S/NF/SS-2) Training and Operational Qualifying Phase: 


(C) There are two other aspects of this program that will require evalu- 
ation by the Project Manager or by his representative; the basic training 
phase and the operational qualifying (or operational training) phase. 


(S/NF/SS-2) The basic training phase consists of a large variety of 
easy-to-verify training tasks that are designed for various stages of 
proficiency. Overall evaluations will be based on the accuracy scale 
used for operational projects and on an additional quantitative 
procedure. The accuracy scale will permit assessment of an individual's 
progress through the six training stages. The quantitative procedure 
(i.e., concept/element analysis) will allow assessments of degree of 
accuracy and will provide the basis for determining what general classes 
of operational projects a viewer can best perform. 


(S/NF/SS-2) The operational qualifying phase follows satisfactory 
completion of the training phase. This phase consists of tasks that 
simulate operational projects but differ in that ground truth can be 
readily determined, and there are no operational consumers. The main 
purpose of this phase is to systematically quantify the operational 
readiness of each viewer for a wide variety of potential operational 
projects. Data from this phase will also help identify the best types of 
operational tasks for remote viewing (RV) applications. 


(S/NF/SS-2) Each RV source will perform several hundred operational 
qualifying tasks per year. These tasks will include representative 
examples from DIA's intelligence production codes and from other 
potential operational requirements. Evaluation will be performed by the 
Project Manager or his representative and will be based on the accuracy 
scale used for operational projects. In addition, accurate estimates of 
operational reliability and utility will be made using quantitative 
methods (i.e., concept/element analysis, pattern evaluation). These 
evaluation procedures will require extensive use of a dedicated data base 
management system for storing large amounts of information and for 
performing appropriate analyses. 


(S/NF/SS-2) It is anticipated that a limited operational capability will 
be achieved by mid-1987. Some operational projects will be initiated in 


FY 1986; however, the type of task accepted will depend on the nature of 
the proposed task and on capability levels of available RV sources. 
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6. Operational Activity (U): 


a. Tasking (U): 


(S/NF/WN) All operational tasks will be reviewed and prioritized by 
DIA/DT. These tasks can originate from DIA/DT and from elsewhere within the 


intelligence community or from other potential users. “Although the mechanism 


for generating/receiving potential tasks from users has not yet been 
finalized, it is anticipated that the Intelligence Community Task Coordinating 
Group may serve this role. These potential tasks would then be transmitted to 


DIA/DT for disposition. 


(S/NF) There is an important condition for accepting potential 
operational tasks: It is that feedback data must be provided to DIA/DT so 
that adequate evaluation of results can be performed. For sensitive projects, 
special caveats and controls will be established. This procedure will assure 
the user community that details of the sensitive task will only be divulged to 


those who have the need to know. 


b. Performing Operational Tasks (U): 


(1) Task Management (U): 
(S/NF) To facilitate optinum accomplishment of all routine and quick 


reaction operational tasks, DIA/DT wil} establish a detailed procedure to 

insure that al] operational needs can be immediately resolved and implemented. 

Activities/functions addressed would include: 

: - 24 hr. availability of management approval authority 
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- 24 hr. avatlability of people who can perform 


operational tasks (viewers, interviewers); 


- Resource allocation authority (people required, 


specific assignments, travel); 


- Procedures for interface with the user community (for 


intelligence and outside); 


- Procedures for interface with others (f.e., VO, HUMINT 


Comittee); 


- Methods/procedures for transmitting RV data to users 


(i.e., spectal courier); 


- Procedures for preliminary evaluation of data and 


decision-making for follow-on actions; 


- Procedures for maintaining security controls, and 


safeguarding viewer identity; 


These and others would be constantly refined to improve over-all 


responsiveness and effectiveness of operational activities. 
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(2) (S/NF/WN) Methodology: A consistent and scientifically-valid approach to 
methods and procedures for active conduct of operational sess{ons will be 
developed. These will be established as firm project requirements, and any 
exception (i,e., due to error, unforseen circumstances) will be clearly noted. 
The user community personnel involved in operational tasks (i.e., as 
observers, or for data analysis) will -be informed of all methods/procedures in 
advance of the conduct of any operational task. This is necessary for 
operational consistency, for data evaluation, and for making decisions on how 
best to apply the RV data. Specific methods/procedures will be devel oped -for 
each operational task in accordance with the nature of the task (i.e., QRT, 
long-term), type of targeting methods required, and the type of training 


program used for developing the viewers’ capability. 
(3) Records (U): 


(S/NF)} Detailed records will be maintained for a1] RY operational 
sessions. Specific data will include people involved, chronology of events, 
data provided to the viewer, targeting methods used, degree of knowledge of , 
target information by those involved in the session, or any other issue 
pertinent to performing the session and evaluating the results. Al] these 
factors will be identified and clear documentation procedures will be 
established prior to initiation of any operational project. The actual RV 
data will also be carefully recorded and controlled, and will be entered into 
the appropriate data base. Data format will include verbal records (tape 
recordings) as well as raw data (written description, sketches). These 
records will be stored in such a way as to protect identify of the user and 
the viewers involved, and will be subject to al7 other security controls 
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available to any authorized individual or project review group as necessary, 
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c. Evaluation of Data (U): 
- 

(S/NF/WN) A data package with pertinent raw RY data, and a summarized 
version performed by project personnel, will be returned to the task 
originator and the IC Task Coordinating Group via DIA/DT. All session methods 
and conditions will be included with this data package. The task originator 
wilt provide an evaluation of the data according to pre-established criteria. 
The originator will return this evaluation to DIA/DT along with sufficient 
background and feed-back data so that the basis of the evaluation can be 
clearly understood. Any difference of opinion will be discussed with the task 
sriginater and noted in the project records. In cases where evaluation fs 
only partial (i.e., for tasks that can’t be immediately evaluated) the file 
will be kept active until such @ time when sufficient data becomes available. 
The task originator will also indicate how the data could be (or was) 
utilized, and will describe the impact such data had on resolution of the 
specific problem of interest. Additional data analysis wil) be performed on 
the raw data in the growing data base to evaluate trends, patterns, 
statistical implications, operational constraints, reliability/utility, and 


other aspects pertinent to data utlization. 


d. Reporting of Results (U): 


(S/NF) In addition to data reports provided to the users, over-all 
project reports will also be prepared on a periodic basis. They will contain 
general information on types of tasks, over-all results, summaries of task 


originator evaluations, statisitics/trends, and other relevant data as 
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required by DIA/DT. These reports would not contain sensitive data; however 
such data could be provided separately on a strict need-to-know basis pending 


appropriate data release approval. 
7. Process Flow Chart (U): 


~ (C/NF) All project procedures and operational activity identified in 


the section for tratning~and for operational projects will be developed in 


detail in a Project Operations Manual. A process flow chart will also be 


prepared. 
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